1. Field of the Invention
The present invention relates to infrared detection systems. More specifically, the present invention relates to systems and methods for providing dynamic range control and non-uniformity calibration of staring infrared imaging systems.
2. Description of the Related Art
Infrared imaging systems have achieved a wide level of deployment in industrial, commercial and military applications. The leading edge of technological development has typically been in the military arena as the ability to generate infrared images has many advantages in both tactical and strategic applications. This is particularly true in airborne systems.
Forward looking infrared (hereinafter xe2x80x9cFLIRxe2x80x9d) systems have been used in airborne applications for many years. Such systems typically employ a focal plane array (xe2x80x9cFPAxe2x80x9d) of infrared image sensing pixels that are coupled with an optical system which views a scene through an entrance pupil of the FLIR system. The FPA is typically housed within a cooled dewar vessel that serves to reduce thermal noise and improve the signal to noise ratio of the system. Infrared energy from a viewed scene is directed to and focused on the FPA through an optical system that may include both refractive and reflective optical components. The optical components in the FLIR system are located between the entrance pupil and a cold aperture of the dewar. Thus, the optical system operates at ambient temperatures that may fluctuate during operation of the FLIR system. Therefore, the optical components themselves are a source of noise to the FLIR systems.
Another limitation on the optical performance of a typical FLIR system is due to the fact that the optical components do not have perfect transmissivity.
As infrared energy is incident upon the FPA, it is integrated over time to produce frames of a video image. In a typical application, with real-time video performance, integration times are on the order of 10 to 20 milliseconds. The video frames generated are conditioned and provided to a video display for viewing by the operator of the FLIR system. The output of each pixel of the FPA is an analog signal proportional to the amount of light energy incident upon that pixel over the integration time period. For a variety of reasons, the output signals from all the pixels in the FPA are not identical, even given a uniform scene.
In typical applications, the analog signal output from the FPA are digitized and changed to a standard video format, such as the EIA RS-170 video standard. One of the principle functions of the FLIR system is to provide a detailed display of the infrared energy emitted from a scene to the pilot. The range of infrared energy levels emitted is very large. These energy levels can be thought of as visible light brightness, especially at the point where the pixel appears on the FLIR display screen for viewing. For example, a FLIR system could be directed to view the ground scene on a cold winter night at northern latitudes where the ground temperature is below zero degrees Fahrenheit. As an alternative, the scene could be a hot desert with battlefield emissions sources, such as fire and explosions. Since staring sensors are DC coupled devices which monitor infrared energy levels (photon flux) varying exponentially with temperature, it should be appreciated that given a range of sensor output levels as they are correlated to scene energy levels, there are problems and opportunities to adjust the sensor dynamic range to accommodate the scene""s dynamic range. Sensor inputs that are too low result in noisy output while inputs that exceed the sensor""s dynamic range result in saturated video.
It is known in the art to employ multiple dynamic range maps between ranges of scene infrared energy levels to the sensor levels. Sensor brightness levels are referred to as xe2x80x9cbucket fillxe2x80x9d levels by those skilled in the art. Bucket fill level, or xe2x80x98BFxe2x80x99 for short, is defined herein as the ratio of the sensor""s digital output (digitized but not yet processed as video) to the maximum digital output of an analog-to-digital converter. For example, in a relatively warm environment, a first range of bucket fill (BF) values will be mapped to a range of scene energy levels. On the other hand, in the case of a relatively cool scene, a lower range of bucket fill (BF) levels are mapped to scene energy.
Changing the dynamic range of the sensor is equivalent to changing the gain (a combination of gain state and/or integration time) and will result in a remapping of BF for the scene. If one switches the sensor to a lower dynamic range (higher gain) while viewing the relatively cool scene, then the BF will rise in proportion to the gain.
In another aspect of prior FLIR systems, it is understood that the FPA sensor array and related control circuitry and sensor circuitry must be calibrated in order to assign correction factors for pixel to pixel responsivity equalization, and pixel to pixel level equalization. Responsivity is defined as gain and is usually expressed in delta mV of sensor output for a 1 Kelvin change in scene temperature (assuming blackbody scene).
As noted above, given a change in uniform scene energy level, not every pixel in the FPA will yield the same change in output signal level. This discrepancy is corrected for during calibration by directing the entrance pupil of the FLIR system toward a uniform thermal body (known to the art as a thermal reference source xe2x80x9cTRSxe2x80x9d) and then calibrating the pixel to pixel responsivity using a responsivity equalization (xe2x80x9cRExe2x80x9d) process. This operation is typically accomplished using specialized hardware adapted to this particular function. The RE calibration is a two-point calibration because gain must be calculated by looking at the difference of two uniform scenes- first near the lower temperature/BF limit of the dynamic range, and a second near the higher temperature/BF limit of the dynamic range. Having this calibration and given a uniform change in scene temperature, the change in sensor brightness levels will be the same for all pixels. The resultant RE set of correction coefficients are gain multipliers for each pixel. One RE set may be used to cover multiple dynamic ranges.
The responsivity equalization eliminates the detector cold shield effect (center pixels will be: brighter without this calibration) as well as FPA pixel to pixel gain variation.
In addition to the RE calibration, typically pixel to pixel level equalization (xe2x80x9cLExe2x80x9d) calibration is employed for each dynamic range. The LE serves to equalize the level factor applied to each pixel in the FPA, and this function is also provided by specialized hardware, and requires that the entrance pupil of the FLIR system be directed to the TRS so that a scene of uniform energy emission is to be viewed during the calibration process. The appropriate RE set for the dynamic range must be used during LE calibration and the TRS temperature must be adjusted to achieve a BF between the high and low BF values of the RE calibration (which produces the RE set being used). A single LE calibration for each dynamic range is typically employed. The resultant LE set of coefficients are level adders for each pixel. The level equalization is required to eliminate fixed patterns created by the optics as well as FPA pixel to pixel level variation. The combination of RE and LE calibrations are required to achieve good uniformity for uniform scenes that (in combination with the optics temperature) produce BF values between the low and high BF values of the RE calibration.
During the course of operation of a FLIR system, even though the system may have been calibrated prior to operation, the scene and/or optics temperature can change. The change frequently pushes toward a bucket fill limit of the previously selected dynamic range, and it therefore becomes necessary to change the sensor""s dynamic range (i.e., change detector gain equally across all pixels) so that the scene is properly mapped to the sensor output (i.e., not too low to prevent noisy video and not too high to prevent saturated video). In prior art systems, an in-operation LE recalibration was required. Such a recalibration requires that the average BF of the scene be measured, the dynamic range be adjusted to achieve an appropriate BF (scene target BF) and the FLIR entrance pupil be directed toward the TRS. The TRS is driven hot or cold until the scene target BF level is achieved, then, the calibration occurs. The calibration procedure necessarily takes the FLIR system off-line for a period of time. Inasmuch as the amount of power available for the TRS is limited, the capabilities of the TRS are limited as well and given that the hardware performing the calibration may take a while, it necessarily requires a significant amount of time for the calibration operation to occur. This is undesirable, especially during operation since the vital function of the FLIR system is interrupted.
Thus, there is a need in the art for a method and system to provide FLIR system calibration across several dynamic range settings that does not interfere with normal operation of the system.
The need in the art is addressed by the method and system of the present invention. The present invention provides a method of maintaining uniformity in a staring FLIR display appearance over time in the presence of changing temperature of scenes and optical elements, particularly in a FLIR utilizing a thermal reference source. The, inventive method comprises the steps of defining a plurality of dynamic ranges each covering a specific range of bucket fill (BF) levels when in a certain gain, performing a plurality of pairs of RE calibrations to produce RE sets of pixel gain corrections during a one time FLIR initialization procedure for each hardware set, and calibrating a plurality of LE sets (one for each anticipated dynamic range and using the appropriate RE set) at each power-up during a FLIR initialization procedure. The algorithms used require that the BF values of each two point RE calibration are centered in and span across most of the corresponding dynamic range/s and that the BF level during calibration of each LE set must fall within its corresponding RE set BF range (i.e., between the BF levels of the two point RE calibration). Then, as needed switching from a present dynamic range to a an adjacent LE calibrated dynamic range during operation according to scene/optics temperature changes that produce BF levels beyond predetermined BF levels for the present dynamic range, thereby maintaining the sensor within a dynamic range adapted to the actual scene/optics temperature ranges (i.e., preventing saturation at very high BF or excessive temporal noise at very low BF and maintaining low fixed pattern noise which is known as uniformity).
In a refinement of this method, the adjacent high and low BF levels of the present and adjacent dynamic ranges overlap (i.e., for the same scene, the BF obtained in either dynamic range is within the range of acceptable BF levels for these dynamic ranges), thereby providing a hysteresis effect against switching frequently between adjacent dynamic ranges. The combination of LE initialization calibrations and automatic dynamic range adjustment eliminates the need for in-flight touch-up calibration. Uniformity in-flight is maintained as long as the BF encountered during operation and the BF obtained during calibration of the LE set for the dynamic range used is between the low and high BF levels obtained in the calibration of the corresponding RE set. Thus, the only limitation of this method is the inability of the low and high BF levels of the RE calibration to span the corresponding dynamic range.
Under normal to cool lab conditions, the TRS temperature and gain (gain state and integration time) are adjusted to allow the BF levels of the RE calibration to nearly span the dynamic range (some reduction in span is allowed only for the coldest dynamic range). In the case that the two point RE calibration can not span across most of the, dynamic range (i.e., a degraded calibration) due to TRS and physics constraints when the FLIR is hot, an alternate RE scheme is used spreading the two BF points as far apart as is reasonably possible along with a warning system to recommend redoing of RE calibrations when the FLIR is cooler.
Defining the dynamic ranges requires two steps: 1. Basic design for nominal case. 2. Refinement during one time initialization. First in the design phase, the number of dynamic ranges and detector gain settings (gain state and integration time) for each dynamic range must be defined. The BF range for each dynamic range must be determined: along with the corresponding BF pair for the two point RE calibration (one RE set may cover more than one dynamic range). It is important that the BF values of the two-point RE calibration span across most of the dynamic range. The BF range for each dynamic range is determined by the number of dynamic ranges, the gain settings of each dynamic range, the expected temperature range of scene and optics, the optics emissivity, the optical gain and most importantly by the need to provide proper scene temperature overlap between adjacent dynamic ranges (providing hysteresis to prevent frequent switching between adjacent dynamic ranges).
The second dynamic range refinement step during a one-time initialization gain calibration is required to determine exact values for BF ranges (high and low values for: dynamic range, RE calibration, and dynamic range switch points used in determining initial dynamic range for LE initialization calibration). This refinement is required in order to adjust the process to take into account hardware tolerances which deviate from the nominal design parameters. The gain of each gain state and the BF corresponding to no photon flux are measured/calculated during the gain calibration. A means is provided for checking if the fluctuation in BF monitored in the gain calibration (each BF is read twice) will result in an inaccurate value of the calibrated gain in each gain state. Also, the calculated values of the gain of each gain state and the BF corresponding to no photon flux are checked against the expected nominal design values and deviations larger than expected are flagged as possible hardware problems.
The present invention teaches a method of maintaining display uniformity in a staring FLIR at the proper sensor dynamic range without the need for in-flight touch up calibration. The first step of this method occurs during a one time initialization defining all required low and high bucket fill values, and performing a plurality of pairs of responsivity equalization calibrations covering the dynamic ranges. High and low BF values must be defined for each dynamic range, RE calibration and dynamic range switch point used in determining the initial dynamic range for LE initialization calibration. Determining the exact values of all high and low BF levels is accomplished by a detector gain state calibration (or gain calibration for short). In this gain calibration the entrance pupil of the FLIR is aligned to a thermal reference source and change in BF is monitored in each gain state as the thermal reference source is heated. Special attention is given to set the integration time properly to prevent saturation in the highest gain state. In this process the BF corresponding to no photon flux and the gain of each gain state is calibrated and then this information is used to reset from the nominal design values all the low and high BF levels.
During the one time initialization procedure after the gain calibration, the RE bucket fill levels are achieved by aligning the entrance pupil of the FLIR to a thermal reference source, driving the thermal reference source to a low thermal level and changing the gain state and integration time as needed to match a low RE bucket fill point. Limits are placed on how low the gain may drop to prevent sacrificing the high RE BF point. A LE calibration is performed and then the TRS is driven to a high thermal level matching a high RE bucket fill point (gain state and integration time remain unchanged). The RE calibration is then performed completing one pair of RE calibrations. The process is performed initially to create the RE set that is used with the lowest dynamic range and then repeated to create RE sets used on successively higher dynamic ranges until all RE sets are calibrated. This order was done with the assumption that optics heat up with time (i.e., harder to later achieve the lowest BF) and that the TRS may be driven cold during cooldown of the detector to save time (also heating the TRS is quicker than cooling). Also, to save time the gain calibration may be done right after the LE low bucket fill calibration for the coldest dynamic range (when TRS is at its coldest and the TRS temperature will be raised for the gain calibration and the RE high bucket fill calibration for the coldest dynamic range).
The next step of this method is to calibrate a level equalization for each anticipated dynamic range at each power-up initialization (i.e., calibrate dynamic ranges which may be used during the mission). During the LE calibration, the entrance pupil of the FLIR is aligned to a thermal reference source which is adjusted to the proper temperature in combination with adjusting the dynamic range to obtain a bucket fill that falls between the corresponding pair of responsivity equalization bucket fill levels. The LE calibration is then performed. LE initialization calibrations starts with the lowest dynamic range (with the TRS as cold as possible) and continues to successively higher dynamic ranges (with TRS adjusted hotter) until all anticipated dynamic ranges are covered (order chosen again assumes optics heat up with time). An attempt is made to center the BF in the second dynamic range setting, but if the TRS requires excessive slewing then the calibration may be skewed to a lower BF. After the second dynamic range, the rest of the anticipated dynamic ranges are calibrated with a BF towards the lower end of the dynamic range, but still above the low BF of the corresponding RE set. Note that calibrating toward lower BF saves time and places the calibration in a regime that one would encounter more often (i.e., when switching to a higher dynamic range the BF will first be at the low end of the dynamic range).
The third step involves choosing the initial dynamic range (with its associated LE set, RE set and gain) based on the unpowered TRS and present scene/optics temperature. In the final step, during an operational time period immediately following initialization, as needed the algorithm changes to another of the plurality of dynamic ranges if the scene/optics temperature results in a bucket fill that exceeds threshold levels (too low or high) of the initial dynamic range. The BF of the sensor is constantly monitored with filtering so that very temporary changes in the scene caused by movement of the gimbaled telescope will not result in unnecessary switching of dynamic ranges. As a refinement of this method, adjacent high and low BF levels of two adjacent dynamic ranges overlap, thereby providing a hysteresis effect against switching frequently between adjacent dynamic ranges.
Redoing of the whole process above may be required. In the case that the two point RE calibration can not span across most of the dynamic range due to hardware and physics constraints when the FLIR is hot, an alternate RE scheme is used spreading the two BF points as far apart as is reasonably possible. The alternate RE scheme uses a simple algorithm which reduces the span of each two point RE calibration and allows for lower gain adjustment in order to achieve this reduced span. Original RE calibrations will check the need for and flag the use of the alternate RE scheme based on the BF obtained when the TRS is at its coldest. The degraded calibration may not adequately cover the range of BF levels encountered in operation and in extreme cases the BF during LE initialization may not fall between the BF levels of the two point RE calibration. A warning system was thus developed so that in cases of a degraded RE calibration, a recommendation to redo the RE calibrations is made when the FLIR is cooler. In the warning system, a RE quality factor is calculated. Since the spread in BF in the RE calibration is dependent upon the BF obtained when the TRS is cold, the RE quality uses this BF value which is obtainable at initialization before the start of RE or LE calibration.